Researchers from the University of Pennsylvania have revealed that AI-generated text detection tools are not as reliable as some people think. These tools are used to analyze texts to see if they were written by AI, but they rely on specific markers that can also appear in human-written texts.
Sometimes, detection tools can easily identify AI-generated texts, but they have difficulty handling some texts such as news articles.
Some of these tools even classify human-written texts as AI-generated.
In their study, the researchers proposed a new approach to generalizing AI-generated texts by providing detection tools with a massive dataset of 10 million diverse documents including news articles, blogs, and recipes.
The detection tools will then be ranked in a public list according to their performance on that data. The researchers explained that they aim to establish new standards for detection tools so that it can be verified that text is indeed written by a human.
Since the launch of GPT-2 in 2019 and GPT-3 in 2022, many issues have emerged regarding AI-generated texts.
Many teachers have expressed concern about students using these templates to write assignments and academic papers.
Some detection tools claim to have 99% accuracy, which is hard to believe. Some even say that AI-generated text detection has become complicated.
The researchers noted that the detection tools could be easily fooled if some words were replaced or a British accent was used.
Also, some tools work best with the models they were trained on, which means they may fail to detect text written with other models.
Additionally, detection tools specifically designed to detect news articles may have difficulty handling recipe text.
In short, as large language models evolve, detection tools face increasing challenges in accurately identifying AI-generated texts.
